ScalarFn validity#8514
Conversation
Signed-off-by: "Nicholas Gates" <nick@nickgates.com> Signed-off-by: Nicholas Gates <nick@nickgates.com>
Initialize direct encoding benchmark sessions with their crate-level kernel registrations so they exercise the session execute-parent path instead of fallback materialization. Cache the ArrayKernels handle in ExecutionCtx to avoid repeated session lookups while trying execute-parent kernels. Signed-off-by: "Nicholas Gates" <nick@nickgates.com> Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: "Nicholas Gates" <nick@nickgates.com>
…ent-kernels Signed-off-by: Nicholas Gates <nick@nickgates.com> # Conflicts: # encodings/bytebool/src/kernel.rs # vortex-array/src/arrays/bool/vtable/kernel.rs
Avoid the per-execution parent kernel cache now that ExecutionCtx already holds the session snapshot, and make patch index lookup use the primitive fast path directly. Signed-off-by: "Nicholas Gates" <nick@nickgates.com>
Store vortex-array's built-in kernel registry on ArraySession so sessions that install the built-in array encodings also get their matching execute-parent kernels. Signed-off-by: "Nicholas Gates" <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
…alidity Signed-off-by: Nicholas Gates <nick@nickgates.com> # Conflicts: # vortex-array/src/scalar_fn/erased.rs # vortex-array/src/scalar_fn/fns/binary/mod.rs # vortex-array/src/scalar_fn/fns/cast/mod.rs # vortex-array/src/scalar_fn/fns/dynamic.rs # vortex-array/src/scalar_fn/fns/get_item.rs # vortex-array/src/scalar_fn/fns/is_not_null.rs # vortex-array/src/scalar_fn/fns/is_null.rs # vortex-array/src/scalar_fn/fns/list_contains/mod.rs # vortex-array/src/scalar_fn/fns/literal.rs # vortex-array/src/scalar_fn/fns/root.rs # vortex-array/src/scalar_fn/typed.rs
Merging this PR will not alter performance
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Simulation | decompress_rd[f64, (10000, 0.01)] |
108.9 µs | 135.1 µs | -19.44% |
| ❌ | Simulation | decompress_rd[f64, (10000, 0.0)] |
108.9 µs | 135.1 µs | -19.4% |
| ❌ | Simulation | decompress_rd[f32, (100000, 0.0)] |
496.1 µs | 583.8 µs | -15.02% |
| ❌ | Simulation | decompress_rd[f64, (10000, 0.1)] |
109.2 µs | 125.5 µs | -13% |
| ❌ | Simulation | patched_take_10k_adversarial |
259.7 µs | 289.1 µs | -10.16% |
| ❌ | Simulation | patched_take_10k_contiguous_patches |
259.6 µs | 288.9 µs | -10.14% |
| ⚡ | Simulation | bitwise_not_vortex_buffer_mut[128] |
244.4 ns | 186.1 ns | +31.34% |
| ⚡ | Simulation | bitwise_not_vortex_buffer_mut[1024] |
304.7 ns | 246.4 ns | +23.68% |
| ⚡ | Simulation | chunked_varbinview_into_canonical[(1000, 10)] |
205.6 µs | 168.9 µs | +21.74% |
| ⚡ | Simulation | decompress_rd[f64, (100000, 0.1)] |
1,020.9 µs | 842.8 µs | +21.13% |
| ⚡ | Simulation | bitwise_not_vortex_buffer_mut[2048] |
427.8 ns | 369.4 ns | +15.79% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing ngates/scalar-fn-validity (ca773ed) with develop (97850e9)1
Footnotes
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
Benchmarks: PolarSignals ProfilingVortex (geomean): 0.933x ➖ How to read Verdict and Engines
datafusion / vortex-file-compressed (0.933x ➖, 2↑ 0↓)
No file size changes detected. |
Benchmarks: FineWeb NVMeVerdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (0.980x ➖, 0↑ 0↓)
datafusion / vortex-compact (1.008x ➖, 0↑ 0↓)
datafusion / parquet (1.011x ➖, 0↑ 1↓)
duckdb / vortex-file-compressed (0.992x ➖, 0↑ 0↓)
duckdb / vortex-compact (1.015x ➖, 0↑ 0↓)
duckdb / parquet (1.002x ➖, 0↑ 0↓)
File Size Changes (1 files changed, +0.0% overall, 1↑ 0↓)
Totals:
|
Benchmarks: TPC-H SF=1 on NVMEVerdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.056x ➖, 1↑ 9↓)
datafusion / vortex-compact (1.042x ➖, 1↑ 1↓)
datafusion / parquet (1.079x ➖, 0↑ 7↓)
datafusion / arrow (1.133x ❌, 0↑ 14↓)
duckdb / vortex-file-compressed (1.077x ➖, 0↑ 4↓)
duckdb / vortex-compact (1.071x ➖, 0↑ 4↓)
duckdb / parquet (1.050x ➖, 0↑ 3↓)
duckdb / duckdb (1.051x ➖, 0↑ 0↓)
File Size Changes (10 files changed, +0.3% overall, 6↑ 4↓)
Totals:
|
Benchmarks: TPC-DS SF=1 on NVMEVerdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.011x ➖, 1↑ 1↓)
datafusion / vortex-compact (0.993x ➖, 4↑ 1↓)
datafusion / parquet (1.015x ➖, 2↑ 4↓)
duckdb / vortex-file-compressed (1.000x ➖, 0↑ 2↓)
duckdb / vortex-compact (1.002x ➖, 1↑ 3↓)
duckdb / parquet (0.994x ➖, 2↑ 2↓)
duckdb / duckdb (1.009x ➖, 1↑ 2↓)
File Size Changes (7 files changed, -0.0% overall, 2↑ 5↓)
Totals:
|
Benchmarks: FineWeb S3Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (0.900x ➖, 1↑ 0↓)
datafusion / vortex-compact (0.975x ➖, 0↑ 0↓)
datafusion / parquet (0.926x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (0.980x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.971x ➖, 0↑ 0↓)
duckdb / parquet (0.981x ➖, 0↑ 0↓)
|
Benchmarks: Statistical and Population GeneticsVerdict: No clear signal (low confidence) How to read Verdict and Engines
duckdb / vortex-file-compressed (1.002x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.997x ➖, 0↑ 0↓)
duckdb / parquet (1.004x ➖, 0↑ 0↓)
File Size Changes (1 files changed, +0.0% overall, 1↑ 0↓)
Totals:
|
Benchmarks: TPC-H SF=10 on NVMEVerdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (0.960x ➖, 1↑ 0↓)
datafusion / vortex-compact (0.989x ➖, 1↑ 0↓)
datafusion / parquet (1.006x ➖, 0↑ 0↓)
datafusion / arrow (0.994x ➖, 0↑ 1↓)
duckdb / vortex-file-compressed (1.001x ➖, 0↑ 0↓)
duckdb / vortex-compact (1.003x ➖, 0↑ 0↓)
duckdb / parquet (0.999x ➖, 0↑ 0↓)
duckdb / duckdb (1.005x ➖, 0↑ 0↓)
File Size Changes (27 files changed, -0.1% overall, 12↑ 15↓)
Totals:
|
Benchmarks: Random AccessVortex (geomean): 0.964x ➖ How to read Verdict and Engines
unknown / unknown (0.979x ➖, 2↑ 0↓)
|
Benchmarks: Clickbench on NVMEVerdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (0.999x ➖, 2↑ 1↓)
datafusion / parquet (1.015x ➖, 1↑ 3↓)
duckdb / vortex-file-compressed (1.016x ➖, 0↑ 3↓)
duckdb / parquet (1.005x ➖, 0↑ 0↓)
duckdb / duckdb (1.008x ➖, 0↑ 2↓)
File Size Changes (103 files changed, +0.0% overall, 62↑ 41↓)
Totals:
|
Benchmarks: TPC-H SF=1 on S3Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (0.892x ➖, 0↑ 0↓)
datafusion / vortex-compact (0.939x ➖, 0↑ 0↓)
datafusion / parquet (0.774x ➖, 7↑ 0↓)
duckdb / vortex-file-compressed (1.003x ➖, 0↑ 0↓)
duckdb / vortex-compact (1.077x ➖, 0↑ 0↓)
duckdb / parquet (0.923x ➖, 0↑ 0↓)
|
Benchmarks: Appian on NVMEVerdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (0.901x ➖, 5↑ 0↓)
datafusion / parquet (0.937x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (0.957x ➖, 0↑ 0↓)
duckdb / parquet (0.969x ➖, 0↑ 0↓)
duckdb / duckdb (0.924x ➖, 0↑ 0↓)
File Size Changes (4 files changed, -0.1% overall, 1↑ 3↓)
Totals:
|
Benchmarks: TPC-H SF=10 on S3Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (0.910x ➖, 0↑ 0↓)
datafusion / vortex-compact (1.093x ➖, 0↑ 3↓)
datafusion / parquet (0.913x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (1.024x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.939x ➖, 0↑ 0↓)
duckdb / parquet (1.104x ➖, 0↑ 1↓)
|
Benchmarks: CompressionVortex (geomean): 0.990x ➖ How to read Verdict and Engines
unknown / unknown (0.962x ➖, 22↑ 7↓)
|
|
This PR has been marked as stale because it has been open for 14 days with no activity. Please comment or remove the stale label if you wish to keep it active, otherwise it will be closed in 7 days |
|
This PR was closed because it has been inactive for 7 days since being marked as stale. |
Makes validity behavior mandatory for ScalarFnVTable so we do not have the odd slightly recursive is_not_null(self) execution using a LEGACY_SESSION.